Dl *' 

VAL 

WON .. 



' ‘''"OX UP*' *y 



ORA -ATE SCHOOL 

OA o Y. .. i ji 



Copyright 

by 

Mark Edward Wiersma 



1995 



STANDARDS AND BENCHMARK TESTS 



FOR EVALUATING LARGE SCALE MANIPULATORS 
WITH CONSTRUCTION APPLICATIONS 

by 

Mark Edward Wiersma 

i 7 

Report 

Presented to the Faculty of the Graduate School 
of The University of Texas at Austin 
in Partial Fulfillment 
of the Requirements 
for the Degree of 

Master of Science in Engineering 

The University of Texas at Austin 



December 1995 



C‘t 














Dl r ' 
H VAL 

MOM 1 u. 



. ,vfv l'ATE school 

i.-, CA ». 31 



STANDARDS AND BENCHMARK TESTS FOR 



EVALUATING LARGE SCALE MANIPULATORS WITH 



CONSTRUCTION APPLICATIONS 



Dedicated to the Advancement of Construction Automation 



ACKNOWLEDGMENTS 



I would like to thank my new found friends and colleagues for providing 
guidance, help and companionship over the past year. I particularly want to thank 
my professors for providing the information to make my education possible. And 
finally, Dr. Alfred Traver who made this report a reality so that I could graduate 
on time. To all I say thanks. 



ABSTRACT 



Standards and Benchmark Tests 
for Evaluating Large Scale Manipulators 
with Construction Applications 

by 

Mark Edward Wiersma, M.S.E. 

The University of Texas at Austin, 1995 
COSUPERVISOR: Alfred E. Traver 
COSUPERVISOR: Carl T. Haas 

New applications of large scale manipulators for construction are 
continually emerging and making them multi-functional construction machines. 

This report presents a set of meaningful benchmark tests to gauge the overall static 
and dynamic performance of different large scale manipulators in order to achieve 
a means of relative comparison. The most important static and dynamic 
performance criteria are defined and a method provided for evaluating them. 

The report reviews the application of human factors engineering to large 
scale manipulator acquired by The University of Texas. Furthermore, a method is 
presented to quantify potential improvements to the human-machine interface. 
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CHAPTER ONE 



INTRODUCTION 



1.1 Background 

The construction industry is ready for the introduction of autonomous 
technology. In particular, large scale manipulators, for instance automated back 
hoes and rebar placement machines, have shown great promise in improving 
safety, productivity, quality of construction and environmental impact (Hsieh and 
Haas 1993). The Construction Automation Group at the University of Texas 
focuses on the development of such systems. 

1.1.1 Grove Pipe Manipulator 

The large scale manipulator, hereafter referred to as the Manipulator, 
owned by the University of Texas at Austin, was originally designed by DuPont as 
a crane attachment for the purpose of installing large bore piping. The designs 
were submitted to the Grove Manufacturing Company for construction and the 
“Pipe Manipulator” became a reality in 1980. Glass (Glass 1984) presents a 
complete description of the crane mounted, electrohydraulically controlled device. 

The initial expectations for the Manipulator were high. It was envisioned 
that the Manipulator would replace the smaller, but more versatile, “cherry picker” 
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(15-ton crane) in the erection of large piping systems like those found at large 
chemical processing plants. After its debut, however, the Manipulator proved to 
be slow, difficult to control and uneconomical to operate. As a result, workers 
regarded the Manipulator as inferior to the “cherry picker” and left it abandoned 
on the job-site. Supporters of the Manipulator were reluctant to give up on their 
new apparatus and thus invited the University of Texas to conduct a study to 
assess its merits and deficiencies. Figure 1 . 1 shows the Manipulator mounted on a 
22 ton crane. 




Figure 1.1 - Crane-Mounted Pipe Manipulator 
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1.1.2 Previous Manipulator Testin 2 

Glass (Glass 1984) was the first to conduct a complete assessment to 
ascertain the Manipulator’s strengths and weaknesses. Conducted on the 
construction site of a new chemical plant in southeast Texas, Glass’s investigations 
determined that the Manipulator was plagued with numerous design flaws which 
hindered its mobility, speed and overall economic performance when compared to 
the capabilities of the “cherry picker”. He further concluded that the control 
system was a primary contributing factor by reason of its number of control levers 
and their location, an operator basket fastened directly to the machine Since the 
gravity leveling operator basket was mounted together with the Manipulator, slow 
arm movements were imperative for providing a safe ride for the operator. This 
suggests that the Manipulator was inherently slow for a reason. In addition, the 
eight control levers, necessary for controlling the Manipulator’s eight degrees-of- 
ffeedom, often left the operator hesitating to determine proper maneuvering 
sequences to optimize performance. Amongst Glass’s conclusions are the 
recommendations for removing the operator basket from the support frame and 
replacing the existing control levers with “joy-stick” controls. Future research and 
design would heed these recommendations. 

Hughes (Hughes 1990) hypothesized “that the Manipulator will compete 
with a conventional crane in pipe spool erection cost, by use of a simple improved 
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teleoperator interface for arm control”. Hughes’ research culminated in the 
development of an entirely new man-machine interface with ergosticks serving as 
the controller. In designing a teleoperated controller, he also removed the controls 
from the operator basket allowing the operator to select his position anywhere 
within the range of a tether of control cables (approximately length 100 ft.). 

To validate his hypothesis, Hughes contrived a test scheme fundamentally 
similar to the one used by Glass. Although the test was a simple pick-and-place 
scenario (Fisher 1989), it represented the first attempt at creating a benchmark to 
compare control schemes. Since his tests were performed in a laydown yard with 
inexperienced operators, Hughes compared his data to Glass’ only after applying a 
“time productivity transformation equation” (Hughes 1990, pp. 120). This 
equation corrected: 

1) The simplification of the pipe rack compared to the chemical plant 
configuration, 

2) The inexperience of Hughes’ volunteers, 

3) The remote location of the control box removed from the operator 
basket. 

In essence, Hughes used Glass’ statistics as a benchmark to conclude whether his 
ergostick interface controller was an improvement over Groves’ levers. He 
concluded it was not. 
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Thomas (Thomas 1995) continued to seek a more advanced controller that 
would increase the Manipulator’s productivity. His research concluded with the 
purchase of a six degree-of-ffeedom optical force and torque sensor Dimension 6 
Geometry Ball manufactured by CIS Graphics, Inc. The new controller was 
installed and tested for functionality. Up to this time no formal benchmark tests 
have been performed. 

In addition to his controller research, Thomas moved the Manipulator into 
the controlled confines of the Construction Automation Lab. His design of a 
“cantilevered space frame” as a new Manipulator mount has made future 
development a more efficient and pleasant task. It is important to notice, however, 
that removing the Manipulator from the crane mount reduces the number of 
degrees-of-freedom from eight to six. Lab tests will be performed without the 
extendibility of the crane, thus reducing the scope of the overall analysis. Figure 
1.2 shows the new Manipulator configuration. 
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Figure 1.2 - Manipulator Mounted on Cantilevered Space Frame 
1.2 Research Objectives 

New applications of large scale manipulators for construction are 
continually emerging and making them multi-functional construction machines 
The objective of this report is to provide a set of meaningful benchmark tests to 
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gauge the overall static and dynamic performance of different large scale 
manipulators in order to provide a means of relative comparison The areas of 
performance measures, human factors, statistical analysis and benchmark tests for 
large scale manipulators are investigated. 

1.2.1 Performance Measures 

Today, a large variety of Manipulators are employed for many different 
uses ranging from the manufacturing assembly line to the construction site. 
Because such a vast assortment of Manipulators with diverse uses and shapes 
exists, standardizing specifications is difficult. However, there are certain 
qualifications which, all else being equal, permit Manipulators of similar type, size 
and function to be compared. This report reviews various methods of 
performance qualification for large scale manipulators for the purpose of 
ascertaining physical specifications. 

1.2.2 Human Influence Analysis 

A common paradox encountered in advancing control technology is the 
development of new systems that can potentially overwhelm their human 
operators. Since the human operator plays a vital role in the performance of the 
Manipulator, human factors cannot be over looked. System efficiency will remain 
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as dependent upon the capabilities of the operator as it does upon the capabilities 
of the Manipulator and its human-machine interface. The application of human 
factors engineering to the Manipulator is examined. In addition, a method to 
quantify potential improvements to the human-machine interface is presented. 

1.2.3 Applying Statistical Analysis 

This report examines the statistical methods for determining the probability 
that the Manipulator will achieve a desired target within a calculated standard 
deviation. The formulas for computing position accuracy, repeatability and there 
respective standard deviations are reviewed for both static and path-related output. 

1.2.4 Benchmark Tests for the Large Scale Manipulator 

This report will define the benchmark test procedures for evaluating the 
performance criteria of large scale manipulators. The performance tests presented 
in this report are based on the American National Standard for Point-to-Point 
(Static) Performance and for Path-Related (Dynamic) Performance evaluations for 
industrial robots and robot systems. The performance criteria are accuracy, 
repeatability, cycle time, overshoot, settling time, relative path accuracy, path 
repeatability, path speed characteristics and cornering overshoot. Methods of 
measuring the performance criteria are not discussed in this report. 
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CHAPTER TWO 



PERFORMANCE MEASURES 



This chapter reviews various methods of performance qualification for 
large scale construction Manipulators for the purpose of ascertaining physical 
specifications. There are several reasons for performance specifications: 

• selecting a machine for a given task; 

• planning work so that tasks fall within the capabilities of a given 
machine; 

• setting goals for new equipment; 

• generating benchmarks to gauge performance parameters of different 
machines, systems and techniques 

Today, a large variety of Manipulators are employed for many different 
uses ranging from the manufacturing assembly line to the construction site. 
Because such a vast assortment of Manipulators with diverse uses and shapes 
exists, standardizing specifications is difficult. However, there are certain 
qualifications which, all else being equal, permit Manipulators of similar type, size 
and function to be compared. The following sections define these qualifications. 



9 



2.1 Static vs. Dynamic Outputs 



Output is the response of the Manipulator to input commands. 

Output is a specified movement or application of force to an object, and can be 
either static or dynamic in nature. 

Static output results when the Manipulator approaches a target point and is 
held in a fixed position. Rebar and pipe placement are just two examples of static 
output. The important performance measure is the final position of the end- 
effector and its payload. In the absence of obstruction, the path followed by the 
end-effector in the performance of the task is irrelevant. Static outputs are much 
easier to measure since the outcome is not in motion. 

Dynamic output, in contrast, results when the Manipulator follows a 
specified path. Evaluation of dynamic output is more difficult because it entails 
continuous measurement of the end-effector position during the execution phase. 
Tracking and surface following are two examples of dynamic outputs. 

2.2 Accuracy 

Accuracy is the measure of the difference between the desired output and 
the achieved output when there is no memory of previously performed tasks. The 
expected error of the output about the mean achieved output when added to the 
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mean accuracy represents some degree of confidence of the accuracy (Colson 
1984). 

There are two types of accuracy, absolute and relative. They differ only in 
their frames of reference. Absolute accuracy is measured relative to the 
Manipulator’s base coordinate system. Figure 2.1 illustrates how absolute 
accuracy relates to the Manipulator’s base coordinate system. 




Relative accuracy is measured from a previously achieved output. Relative 
accuracy occurs when a specified output is planned from some calibration point 
other than the base coordinate system. It is an especially important performance 
measure when working with tool jigs or when working from a benchmark on a 
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construction site. It is assumed that the relative accuracy of the Manipulator is 
adequate enough so that given a calibration point other than the base coordinate 
system, subsequent output can be determined with some level of confidence. 
Relative accuracy is the measure of accuracy of location points within the 
Manipulator’s work space to a calibration point located within the work space. 
Figure 2.2 illustrates the concept of relative accuracy. 




2.3 Repeatability 

Repeatability is the measure of how closely the achieved output clusters 
about its mean. Todd (Todd 1986) illustrates the difference between repeatability 
and accuracy in the Figure 2.3. 
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In the target analogy of Figure 2.3, each dot represents an attempt to 
achieve the desired output, in this case, hitting the cross-hairs on the target. The 
size of the dot cluster represents the measure of repeatability and the closeness of 
the center of the cluster represents the accuracy. It is clear from this figure that it 
is possible to obtain a high level of repeatability without being accurate It is 
important to note the difference between accuracy and repeatability. 
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Repeatability is a very powerful benefit to an autonomous Manipulator. It 
denotes the ability to repeat programmed outputs consistently. Repeatability is 
more valuable than accuracy; if the error is constant, accuracy can always be 
corrected. For these reasons, repeatability is often considered more meaningful 
performance measure than accuracy. 

2.4 Resolution 

The capability of feedback devices (encoders) in coordination with the 
control system in determining the locality of the end-effector and its calibration 
point determines the resolution of the Manipulator. In broader terms, Wodinski 
(Wodinski 1987) defines resolution as the measure of the smallest possible 
increment of change in the variable output of a device or position sensors. 

Although resolution is ultimately determined by the capabilities of the 
system’s actuators and components, to the user it is simply the minimum 
consistently commandable output that is measurable at the end-effector. 

Resolution comprises both position and orientation of the Manipulator’s end- 
effector. 

Resolution has some degree of impact on the teach-and-repeat capabilities 
of the Manipulator, such as teaching the Manipulator to place a cylindrical bar into 
a round hole. If the tolerance of the bar and hole fit is smaller than the resolution 
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of the Manipulator system, then insertion of the bar into the hole may never be 
possible. Fortunately there are many construction applications for which high 
tolerance specifications are not required. 

2.5 Celerity 

There are two primary performance criteria associated with the speed of 
the Manipulator. One is economic and the other kinematic. Performance 
specifications provided by the manufacturer almost always include speed. The 
engineer embarked on the advancement of the Manipulator’s capabilities must 
often evaluate the machine's swiftness and speed in economic terms. Engelberger 
(Engelberger, 1980), a pioneer in robotics research, writes “no matter what the 
social benefits are, no matter how clever the technology, no matter how pretty the 
robot is to watch, every proposed investment in robotics has to pass the test of 
critical financial appraisal.” Perhaps financial performance is the toughest test of 
all. The most brilliant innovations are failures if they lose money or even if they 
cannot provide an attractive return for the investor. In theory, at least all money is 
competing for the highest possible return. 

Glass (Glass 1984) measured the performance of the prototype Grove 
“Pipe Manipulator” by comparing its operating cost efficiency to that of the 
incumbent “cherry picker.” Unfortunately for Grove and DuPont, the “cherry 
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picker” was the better performer and the Manipulator was taken off the jobsite and 
given to the University for further research and development. 

Hughes (Hughes 1989), after devising a new control scheme, proceeded 
with the same economic analysis methods. He knew that replication of the original 
test procedure was essential for comparison and ultimately to validate his 
hypothesis. 

In order to properly evaluate the Manipulator’s improved control 
characteristics, it is essential to relate new benchmark tests to the previously used 
economic ones. This is the only way to quantify performance improvements. 
Hughes accomplished this by applying a “productivity transformation equation” to 
his results to enable a comparison of the present and past. Likewise, such 
transformations offer the opportunity to arrive at the wrong conclusion if not done 
carefully. 

From a kinematic standpoint, both velocity and acceleration will have 
important impacts upon the rate of performance of the Manipulator. Because 
cycle time is of substantial concern for the applications engineer, benchmarks for 
the maximum speed of each degree-of-freedom will be important. Appleton and 
Williams (Appleton and Williams 1987) suggest the following: 

One approach that is useful for comparison purposes is to define a 
test cycle and measure the total travel time. The test cycle should be 
repeated a number of times and a mean and standard deviation determined 
for the cycle time. Within a simple trajectory it might also be useful to 
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know the speed variation, the mean speed and the average speed, all of 
which are useful for building up theoretically predicted cycle times 

Maximum and minimum accelerations will be important for end-effector 

design. This information is necessary for determining the forces required to hold 

items securely to avoid slippage during handling operations. 

Finally, it is important to bear in mind that the speed of the Manipulator 

during unconstrained point to point movement will be faster than during 

continuous path control movements necessary during tracking and surface 

following. The control of speed and the control of position are intimately related 

for path following applications. 

2.6 Overshoot and Settling Time 

In robotics ANSI defines overshoot as the “largest distance of overtravel 
past the target position along the direction of motion after the robot is within a 
settling bandwidth (±S).” S is the standard deviation and is defined later. 
Overshoot is predominate during “violent changes in direction and mass and 
during acceleration and deceleration.” (Wamecke et al. 1985) Cornering 
overshoot is defined by ANSI as the largest deviation outside of the reference path 
after the Manipulator has passed” the corner. 
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Settling time is “a period of time required for the robot to remain within a 
limit ( ±S) from the target point after a move command is executed. Sometimes 
settling time is called oscillation. Settling time is measured as the elapsed time 
starting from initial crossing into the limit band to the last point that is outside this 
limit.” Overshoot and settling time are shown in Figure 2.4. 




Overshoot should be measured to quantify the Manipulator’s capability to 
make smooth and accurate stops at target points. This is an important 
performance measure during applications involving large inertias, high speeds or 
frequent stops. Settling time should be measured to quantify how quickly the 
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Manipulator can stop at a target point. Overshoot and settling time are related 
parameters. Note both overshoot and settling time depend not only on the mass 
distribution of the Manipulator but also on the mass distribution of the payload. 

ANSI/RIA 15.01-1 90 outlines the American National Standards for 
measuring overshoot and settling for industrial robots and robot systems. 
ANSI/RIA 15.01-2 92 outlines the standards for measuring cornering overshoot. 

2.7 Compliance 

Static compliance is the amount of elastic deflection of the Manipulator 
under a static force. ANSI/RIA 15.01-1 90 outlines a method for measuring the 
three principal compliance components along a robot’s base coordinate system. 
Both torsional compliance and coupling effects are excluded 

Dynamic values (frequency, damping, amplitude, phase) are also important 
design criteria. Here modal analysis is a useful tool for Manipulator design. 

2.8 Sensitivity 

Specific sensitivities important to the Manipulator are payload, direction of 
approach, temperature and geometry properties. All of these factors influence the 
accuracy, repeatability and resolution of the Manipulator system. 
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2.8.1 Payload 

Payload refers to the item handled alone and does not include the end- 
effector or arm, which is considered part on the Manipulator. If the arm has 
multiple end-effector attachments, each attachment should be benchmarked 
separately because of weight and functionality differences. The effect of payload is 
complicated by the fact that it depends not only on the mass of the payload but 
also on the mass distribution and orientation. 



The inertia of the Manipulator-payload system will have substantial impact 
upon overshoot and oscillation. System inertia is dependent upon the 
instantaneous payload position or the load and end-effector position at anytime 
given instant. System damping is critical to attaining targets with minimal 
overshoot and settling time. 

Since loading affects speed, accuracy and resolution, various loading 
conditions must be benchmarked. Appleton and Williams (Appleton and Williams 
1987) recommend testing at 0, 50 and 100% of maximum payload and at 20, 50, 
or 1 00% of speed available. 




m 
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2.8.2 Direction of Approach 

One of the hindering design points discovered by Glass (Glass 1984) on the 



original Grove “Pipe Manipulator” was limiting movement parameters. He 
observed that the pivoting, booming, and telescoping characteristics of the 
Manipulator limited its productivity and usefulness. These conditions made the 
Manipulator very sensitive to the direction of approach chosen by the operator. 

Consider putting a round peg in a round hole as illustrated in figure 2.4. 
This figure clearly demonstrates the sensitivity associated with direction of 
approach of a kind encountered in autonomous Manipulators. 




Figure 2.5 - Peg in a Hole Problem (Appleton and Williams 1987) 
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2.8.3 Temperature 

Hydraulic systems, especially those used in heavy equipment, suffer from 
the influence of heat. In hydraulic equipment such as the Manipulator, oil is 
pressurized and released as it circulates through the system. Heat is generated and 
the oil temperature rises. Some of this heat is gradually transferred to the 
structure and joints and serves to raise the temperature of the machine 
components. 

Gradual heat build up affects the feedback system, causing overall 
positional drift. Procedures should be implemented to reach a stable operating 
temperature quickly and remain there. Sufficient time must be allowed for warm 
up before testing commences. Care should be taken to maintain temperatures 
during stoppages. 

2.8.4 Geometry Properties 

Figure 2.5 illustrates that geometry sensitivity often plays a role in 
accentuating or attenuating position errors. Revolute joints such as the one 
depicted in the Figure 2.5 tend to be greater sources of errors than prismatic joints. 
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Figure 2.6 - Sensitivity of Extended Arm (Todd 1986) 
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CHAPTER THREE 



HUMAN INFLUENCE ANALYSIS 



3.1 Human Factors Engineerin 2 

Human factors engineering is the study of new technological products and 
the people who make them work. Adams (Adams 1989) provides a more formal 
definition: 

The field of human factors engineering uses scientific knowledge about 
human behavior in specifying the design and use of a human-machine 
system. The aim is to improve system efficiency by minimizing human 
error. 

The United States government played a key role in the evolution of human factors 
research. During World War II, engineering systems became increasingly complex 
and compelled the government to establish new test centers at Wright-Patterson 
and Brooks Air Force Bases. More recently, human factors engineering, referred 
to by some as ergonomics, has become even more important with more and more 
utilization of state-of-the-art microprocessor control technology. 

Today’s technological advancements continue to expand the performance 
envelope of machines and machine systems at an unprecedented rate. 
Unfortunately, these advancements come with a rather hefty price tag. Moreover, 
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accompanying these high tech and high cost components is the disadvantage of 
complexity. This is important to note since “the need for human factors 
engineering grows in direct relationship to complexity of the man-machine 
systems.” (Olex et al. 1983). 

A common paradox encountered in advancing technologies is the 
development of new systems that can be exceptionally difficult to use and even 
overwhelm their human operators. Since the human operator plays a vital role in 
the performance of the Manipulator, human factors cannot be over looked. 

System efficiency will remain as dependent upon the capabilities of the operator as 
it does upon the capabilities of the Manipulator and its control system. 

Human factors engineering also considers the social and motivational 
issues in the analysis, design, implementation, control and operation of the 
Manipulator. However, these topics are beyond the scope of this report. The 
reader is referred to Graham (Graham 1991), Adams (Adams 1989) or Sanders 
(Sanders et al. 1993) for further information. 

3.2 Human Factors and the Manipulator 

Glass (Glass 1984) observed that the Manipulator’s eight control levers, 
necessary for controlling the Manipulator’s eight degrees-of-freedom, often left 
the operator hesitating to determine proper maneuvering sequences to optimize 
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performance. Consequently, Dupont had a pipe manipulating machine that was 
very difficult to control efficiently. But, more importantly, when compared to the 
less expensive “cherry picker”, the “Pipe Manipulator” was found too 
uneconomical to operate. Both had similar operating costs but the duty cycle of 
the “Pipe Manipulator” was too low and required a large active site to stay busy. 
When not busy moving pipe, the retention of the crane function would have 
increased the “Pipe Manipulator’s” flexibility and usefulness. Although Glass 
observed other design factors that hindered performance of the prototype machine, 
control has been a primary focus for the University of Texas Construction 
Automation Lab. 

Hughes (Hughes 1990) chose the man-machine interface as his topic of 
research since better control would correct “intolerably slow and clumsy 
operation." He envisioned the Manipulator as a tool to be used by a craftsworker 
without any particular heavy equipment training. He also believed that new 
controls would provide a path for continued advancement to higher levels of 
automation. 

The possibility always exists, however, of designing an autonomous 
Manipulator too complex for the average user to efficiently operate. This, of 
course, defeats the purpose of creating a better, more powerful control system. 
Recent undertakings to improve the Manipulator control system include Hughes’ 
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(Hughes 1990) “ergosticks" and more recently, Thomas’ (Thomas 1995) 
Dimension 6 Geometry Ball. These efforts have focused on teleoperated control, 
were the human operator guides the Manipulator through a joystick or similar 
device. Clearly the human operator remains an integral part of the control loop. 

There are a number of human issues that need investigating in order to 
improve the human-Manipulator interface. These include, but are not limited to, 
the following: 

• safety (always the first priority), 

• perceptual limitations of the human operator including visibility and 
dexterity, 

• operator training, 

• software interface design, 

• robustness of design, 

• discriminability of controls and control axes. 

The anticipated outcome of applying human factors engineering is simple. 
When applied effectively it should increase the compatibility of the human operator 
with the Manipulator control system. This approach does not increase the overall 
capability of the system, but makes it more user friendly. The end result is a 
machine easier to operate for the human user. Perhaps then the “cherry picker” 
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will become the obsolete piece of construction equipment Glass (Glass 1984) 
predicted. 

3.3 Human Factors Tests of the Manipulator 

Hughes (Hughes 1990) performed a series of simple tests to validate his 
hypothesis that the Manipulator will compete with a “cherry picker” in pipe 
erection cost. In these tests, a set of operators erected pipe using the Manipulator 
in a simulated plant environment. The experimental variable was task completion 
time. His “ergosticks” open loop rate interface time was the test variable and the 
Grove interface time was the control. Other measurements consisted of all 
operator inputs and Manipulator main boom positions versus time. 

The tests performed were simple but effective. Four inch plastic pipe was 
picked up from a lay down area at ground level and placed on an elevated pipe 
rack. Figure 3.1 illustrates the test layout. 

During the execution of these tests, Hughes made some interesting 
discoveries about the human factors associated with his new controller. He 
explains in detail the problems associated with operator vision during 
teleoperation. Among his findings was that translational and angular alignment of 
the Manipulator jaws from a remote command site had certain affixed human 
limitations. Accurate pick and placement of the pipe was dependent upon the 
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operator’s depth perception at a distance of 40 feet. 
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In addition, general observances regarding the “ergosticks” from the test 
operators summarized repeated confusion between control axes, neutral positions 
were not firm enough and control arms were uncomfortable/wrong height. In 
contrast, the Grove controls seemed complicated and difficult to distinguish during 
lifting operations. He concluded that close attention must be paid to the location 
of the controls and the tasks of the Manipulator operator. It is not surprising that 
these observations are associated with human factors. 

Testing is important for validating and quantifying any design 
improvement. Hughes’ tests were simple to set up and run, and represented a 
typical task regularly performed on construction sites. 

3.4 The Learning Curve 

Nof (Nof, 1985) describes learning “as the process by which the time or 
the cost per cycle decreases as the number of performed, repetitive cycles 
increases." The learning process has been observed in humans to follow the 
typical learning curve given by the equation: 

T(//) = T(l) x n' A 

where: T = time (or cost) per cycle 
T(l) = time (or cost) of the first cycle 
n = the number of cycles performed 
A = an improvement constant, determined by the learning rate. 
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Figure 3.2 shows a typical learning curve. 




Figure 3.2 - The Learning Curve (Nof 1985) 

The main factors in human learning are (Nof 1985): 

1. person’s age; 

2. the amount of previous experience in learning; 

3. personal physical and psychological capabilities; 

4. the job complexity in terms of cycle length, amount of uncertainty, and 
degree of similarity to previous jobs. 

Now we can describe mathematically the learning curve associated with each new 
control system tested. 

Hughes’ recorded the average erection time versus the first, second and 
third pipe spool erected for each interface configuration (Grove and “ergosticks”). 
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Hughes’ Test Results 



Erection 
time (sec) 


Pipe Spool 
Number 


245 


1 — 


240 


2 


202 


3 


300 


1 — 


215 


2 


225 


3 



“Ergosticks” 



Grove 



The graph is shown in Figure 3.3. 




Figure 3.3 - Learning Curve for Hughes’ Pipe Spool Erection Test 
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Note that smoother curves can be attained by obtaining more data and 
curve fitting the results as shown in Figure 3.1. It is clear from this graph that 
both curves are nearly flat. This is an indicator of the difficulty in operating each 
system. 

By performing the identical test on new control systems, we can directly 
compare the curves (old versus new) and quantify, both numerically and 
graphically, the improvement associated with each new control system. The 
“steepness” of the learning curve will be the indicator of the operator’s ability to 
learn the system. The “steeper” the curve the quicker it is to learn. 
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CHAPTER FOUR 



APPLYING STATISTICAL ANALYSIS 



Statistics is a science for analyzing data to ascertain errors, precision, and 
general validity of experimental measurements. One of the major branches of 
statistics is probability. Probability provides tools and methods for describing 
random variations in a system. 

This chapter will examine the statistical methods for determining the 
probability that the Manipulator will achieve a desired target within a calculated 
standard deviation. The formulas for computing position accuracy, repeatability 
and their respective standard deviations will be reviewed for both static and path- 
related output. But first, we shall briefly review the various sources of error that 
hinder performance. 

4.1 Error Types 

Inherent in the Manipulator are various sources of error that adversely 
affect its accuracy and repeatability performance. Improving the Manipulator’s 
performance will require understanding these sources. The factors that influence 
performance are categorized as geometric, nongeometric, or dynamic in nature 
(Hudgens and Tesar, 1992). A brief description of each is given below: 
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Geometric factors are essentially state independent constants that 



define the static input/output relationship between desired output 
coordinates and Manipulator achieved coordinates. These error 
sources include: kinematic parameter error, Manipulator placement 
error, encoder resolution, gear error, etc. 

• Nongeometric factors are dependent upon load and environmental 
conditions and affect the kinematic input/output connection of the 
system. These errors include: compliance, gravity, backlash, 
temperature, etc. 

• Dynamic factors affect the higher order performance of the 
Manipulator and are also state dependent. These errors include: 
inertia, friction, vibration, control system dynamics, etc. 

Together these sources of error make it impossible for the Manipulator to achieve 
desired outputs exactly. 

The robotics community employs several performance enhancement 
solution techniques to improve industrial robot performance. These techniques 
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are categorized as (1) design, (2) sensing, or (3) control (Hudgen and Tesar, 
1992). Design solutions are based on improvements in robot construction such as 
employing higher manufacturing tolerances or designing lighter and stiffer 
components. This is perhaps a long term solution for future Manipulators but not 
applicable to our current version. Sensor enhancement solutions use an external 
sensing system to eliminate error by adjusting joint feedback signals This is very 
expensive and not practical for the Manipulator. Finally, the control enhancement 
solutions typically implement standard independent-axis PID (Proportional 
Integral Derivative ) control strategies. Although this solution is more 
economical, the level of sophistication might prohibit its use on the Manipulator. 

Whether the Manipulator will need a performance enhancement boost will 
depend upon the statistical data collected from performance testing The next 
section describes the statistical analysis presented in ANSI/RIA 15.05-2 90 and 
92. 
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4.2 Statistical Formulas 



The following statistical formulas have been defined by ANSI to quantify 
the static and path performance criteria of Robots. This report will apply the 
same performance criteria to the Manipulator. 

Position accuracy is a statistical measure of the spatial deviation between 
commanded and achieved Manipulator positions. It is the measured difference 
between the commanded pose and the attained pose of the Manipulator. Here 
pose is defined as a position and orientation in space. The magnitude of the 
accuracy deviation (</,) at the /th position is given by: 



where X a „ Y a ■, and Z ai are the coordinates of the attained pose at the /th 
measurement and X z \, 7 C i and Z c j are the corresponding commanded pose 
coordinates. 

4.2.1 Static Accuracy 

The two statistical measures for static accuracy are mean and standard 
deviation. The mean position accuracy (d p J is given by: 

- 1 N 

And its standard deviation (Sp A )is: 
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PA 



r-i^r- 



where N is the total number of measurements. For engineering experiments, 
Holman (Holman 1 984) maintains that it is desirable to use at least 20 
measurements in order to obtain reliable estimates. ANSI suggests at least 50 
measurements for testing robotic systems. 



4.2,2 Path Accuracy 

ANSI defines path accuracy as the measurement of the distance between a 
reference path and any given attained path. Two types of path accuracy, relative 
and absolute, are presented in ANSI/RIA 15.05-2 92. They differ only in the 
definition of the reference path used to compute deviations. However, ANSI 
recommends using the relative path type because this approach simplifies the 
necessary measurement methods. Nevertheless, the following formulas apply 
regardless of which type is used. The relative path accuracy test, and all other 
tests, will be outlined in chapter five. 

The two statistical measures for path accuracy are maximum and average 
deviation. ANSI defines maximum deviation (AC) as the maximum distance 
between any given path and the corresponding reference path. The magnitude of 
the maximum deviation of the worst path (ACrel) is given by: 
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ACrel - max max^(U.,-U, i ) J +(V. i ,.V 4 ) 1 , 
where n is the number of measurement cycles (minimum of 10), (U aj , V ) are the 
coordinates of the attained path and (U^ , V rjj ) the coordinates of the reference 
path for the /th cycle and yth evaluation point. See Figure 4. 1 . 




Figure 4.1 - Path Accuracy Definition (ANSI/RIA R1 5.05-2 92) 
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The average deviation is the average of the distances between any given 
attained path and the corresponding reference path. The magnitude of the 
average relative path accuracy (AC^) is calculated as follows: 



AC ML = ^Z^ivu^+o^-v,,) 1 , 



where (U a V ) are the coordinates defined by: 



U a =~ZU a and 

J n ^ 0 




4.2.3 Static Repeatability 

ANSI defines static repeatability as the measure of deviations between 
achieved output and the mean of the output after commanding the Manipulator to 
the same pose n times from the same direction. This test will measure the 
Manipulator’s unidirectional repeatability. Omnidirectional repeatability is the 
measure of repeatability when the Manipulator approaches from different 
directions. 

Three measurement locations are used in the test and the arithmetic mean 
must be calculated at all three locations before calculating the mean and standard 
deviation of repeatability. The mean is given by: 
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n 




i-l 



n 




i=l 



n 




i=l 



Where (a,b,c), are the three measurement locations for the /th cycle for a total of 
n cycles. ANSI suggests completing five hundred cycles after system stabilization 
has occurred. Seventy-five to one hundred cycles would be more appropriate for 
the Manipulator. 

The deviation (d) for each measurement is determined by the equation: 

d(i,b,c)i — ^(^(a,b,c) i _ -^(a.b.c) ) + ( ^(a.b.c)^ — X>,b,c) ) + (^(a,b,c), — ^(a,b,c) ) • 

Thus, the total mean repeatability (d^p) is: 



n 



n 



n 



2X, + 2X, + 2x. 



1=1 



1=1 



1=1 



3n 



Now, the standard deviation (Srep) can be computed from: 



n 



n 



n 



Xw, '^REP ) + S (^bi " ^REP ) + S (d a - ^rep ) 




i=l 



i=l 



3n - 1 
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4.2.4 Path Repeatability 



ANSI defines path repeatability as the measure of the closeness between 
multiple paths. The statistical data calculated for path repeatability is same as 
relative path accuracy except that dynamic path repeatability uses a reference path 
that is the average of a path traversed n times. ANSI recommends completing a 
minimum of ten (n = 10) path cycles. 

Path repeatability is a scalar value that represents the magnitude of the 
deviations in a given evaluation plane (defined in Section 5.2.7). These deviations 
are measured in the coordinate system of the evaluation plane (see Figure 4.2). 
The path deviation (Aj) is given by: 



And the average path repeatability (PR) is: 

— m 1 ^ 

PR = max -aD- . 

J* 1 n ,J 

Again, m is the number of evaluation points (j), n is the number of measured 
cycles (I) and (U 3j , V a ^ ) are the coordinates defined by: 




The maximum path repeatability (PR) is: 



m n 



PR = max max D„ • 

i=1 i=l 1J 



j=l 1=1 
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Figure 4.2 - Projection of Attained Path onto Evaluation Plane 



43 



4.3 Probability Distribution 



Repeatability is necessary for reliable replication of motion sequences that 
are taught on-line. Repeatability is a relative measure of precision. However, for 
the reasons mentioned earlier, the Manipulator can never be expected to reach the 
target point exactly. The Manipulator actually achieves a range of positions some 
distance from the target. It is reasonable to expect that, over numerous attempts 
to hit a target, the actual output achieved will form a known statistical 
distribution. 

Figure 4.3 illustrates the concept of probability distribution. This figure 
displays how the probability of achieving a desired output is distributed over a 
distance x. Each value of the ordinate p(x) gives the probability that the output 
will succeed in achieving a random position x. Probability distributions are an 
effective tool used to describe repeatability characteristics. 

The most commonly used probability distribution used to interpret 
repeatability characteristics is the normal or Gaussian distribution. The equation 
for this distribution is: 



P(x) = 



_1 

Sjln 



e ^-x) 2 /2S 2 



where X is the arithmetic mean and S is the standard deviation as previously 
discussed. 
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Figure 4.3 - Probability Distribution of Output 
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Figure 4.4 provides further illustration of the importance the probability 
distribution has on characterizing Manipulator output. Firstly, the original point 
taught to the Manipulator will not necessarily be the arithmetic mean. The 
position of the mean will be influenced by the manner in which the target was 
originally taught and later approached. Consequently, two distribution curves 
result from the two different directions of approach. 

Secondly, with the use of probability distributions, it is possible to 
determine the likelihood that certain achieved outputs will fall within a specified 
deviation from the mean of the output. Calculating the probability that an output 
will fall within one, two and three standard deviations (5) of the mean will result 
in 68.27%, 95.45% and 99.73% respectively. Hence, it is possible to 
superimpose positional tolerances such that 99.73% of the output will fall within a 
specified band. Many robot suppliers often quote the width of this band as the 
repeatability of their robot (Appleton 1986). 

And finally, the distance between the LHS and RHS distributions is called 
the mean hysteresis range, and the distance between three standard deviations is 
called the mean position variance. Care must be taken when programming the 
Manipulator to ensure that taught points are approached from the same direction 
(unidirectional repeatability). Use of a safe, approximate stand off point followed 
by a slow approach to the target will reduce the effects of mean position variance. 
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Figure 4.4 - Repeatability Characteristics (Appleton 1987) 
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CHAPTER FIVE 



BENCHMARK TESTS FOR THE LARGE SCALE MANIPULATOR 



The intent of this chapter is to define the benchmark test procedures for 
evaluating the performance criteria of Chapter 2. Methods of measuring the 
performance criteria are not discussed in this report. The reader is referred to 
chapter eight of Knopf (Knopf and Tesar, 1994) for information concerning robot 
metrology equipment. 

The following performance tests are based on the American National 
Standard ANSI/RIA 15.01-1 90 for Point-to-Point and Static Performance and 
ANSI/RIA 15.01-2 92 for Path-Related and Dynamic Performance evaluations for 
industrial robots and robot systems. The static performance criteria are: accuracy, 
repeatability, cycle time, overshoot and settling time. For dynamic performance 
the criteria are: relative path accuracy, path repeatability, path speed 
characteristics and cornering overshoot. ANSI feels that these criteria represent 
the best indication of the overall static and path-related performance of industrial 
robots. 
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5.1 Coordinate Systems 



This section establishes a common set of coordinate systems that describe 
the location of the Manipulator, end-effector and test data. 

5.1.1 World Coordinate System 

This coordinate system establishes a fixed frame of reference and is usually 
used to describe the workcell layout. This is a Cartesian coordinate system that 
consists of three translational coordinates (Xo,Y 0 ,Z 0 ) and three rotational 
coordinates (Ao,B 0 ,Co). The translational coordinates form a right-handed 
coordinate system with the +Zo direction collinear with but opposite in direction to 
the gravity vector. The rotational coordinates are defined such that Ao, B 0 and Co 
rotate about the Xo, Y 0 and Z 0 axes respectively. Note that all rotation 
coordinates follow the right-hand rule convention (see Figure 5.1). 

5.1.2 Base Coordinate System 

The Base Coordinate System is used to establish the location of the 
Manipulator within the World Coordinate System. The Base Coordinate System 
defines the location of the standard test path and provides a frame of reference for 
recording test results. The Base Coordinate System is a Cartesian coordinate 
system that consists of three translational coordinates (Xj,Yi,Zi) and three 
rotational coordinates (Ai,Bi,Ci) comparable to the World Coordinate System. 
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The origin of the Base Coordinate System is located at the intersection of 
the centerline of rotation of the first axis and the floor of the lab. The first axis is 
the first axis of motion encounter starting at the frame and progressing toward the 
end-effector. The +Xj axis points away from the origin and continues through the 
centerpoint of the Manipulator working space (C w ) on the plane defined by the 
interface between the Manipulator’s frame and the floor of the lab (see Figure 5.1). 

The working space is the space in which the Manipulator has no limitations 
in the movement of the mechanical interface other than those imposed by the 
joints. The centerpoint (C w ) is the geometric center of that space (see Figure 5.2). 
The +Zi axis points in the direction of the mechanical structure of the Manipulator 
advancing away from the floor of the lab 

5.1.3 Mechanical Interface Coordinate System 

The Mechanical Interface Coordinate System establishes the location of the 
end-effector relative to the Manipulator position. It is a Cartesian coordinate 
system that consists of three translational coordinates (X m ,Yn„Z m ) and three 
rotational coordinates (An,,Bm,C m ) comparable to the World and Base Coordinate 
Systems. 
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Section A-A 

Figure 5.2b - Reference Center Line E1E2 



The origin of the Mechanical Interface Coordinate System is located at the 
intersection of the Manipulator’s Roll, Pitch and Yaw axes in their mid-positions 
(see Figures 5.1 and 5.3). The +Z m axis is defined by the centerline of the Roll 
axis and points outwardly normal to the mechanical flange of the Manipulator. 

The +X m axis is defined by the centerline of the Yaw axis and points away from 
the +Zi axis. The +Y m direction is defined by the right-hand rule coordinate 
system convention. 



5.1.4 Test Equipment Coordinate System 

The Test Equipment Coordinate System establishes the direction of the 
sensor output. It is a Cartesian coordinate system that consists of three 
translational coordinates (X t ,Y t ,Z t ) and three rotational coordinates (A,,B t ,C t ) 
comparable to the other systems. The (X t ,Y t ,Z,) coordinates form a right-handed 
coordinate system with the origin established by the measurement apparatus. The 
vector relationship between the Base Coordinate System and the Test Equipment 
Coordinate System shall be recorded with the test output. 
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Figure 5.3 - Mechanical Interface Coordinate System 
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5.2 Standard Test Conditions 



ANSI recommends the following conditions under which the standard test 
should be performed. 



5.2.1 Test Environment 

The environment shall be maintained at the following conditions: 

1 . The ambient temperature shall be 18°C to 30°C and be maintained 
within a total range of 2°C. 

2. The relative humidity shall be maintained between 30 and 90%. 

3. The vibration content shall be measured and noted if believed to 
significantly affect the test results. 

These test conditions shall be noted with the test results. 

The intent of these rigid test conditions is to achieve consistent 
performance. It is important to ascertain the Manipulator’s full performance 
potential. Obviously these conditions will not be met during outside crane 
mounted testing. Regardless, the environment conditions shall always be noted 
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5.2.2 Test Load 



The rated payload of the Manipulator is greater than 140 kg and thus falls 
into ANSI Standard Test Load Category 12. For this class, ANSI recommends a 
payload of no less than 50% of the manufacturer’s maximum rated specification. 
Since the Manipulator’s maximum payload is currently 1600 lbs. (725 kg), the 
standard test load shall be no less than 800 lbs. (363 kg). The weight, moment and 
inertial properties of the test load shall be recorded. 



5.2.3 Test Point 

The test point is the physical point on the end-effector where the 
Manipulator position is measured. Per ANSI recommendations, the test point 
shall be located as close as possible to, but not necessarily coincident with, the 
center of gravity of the test load. The axial and radial offset of the test point will 
be greater than or equal to the axial and radial offset of the test load center of 
gravity. The following definitions apply: 

• axial offset. The distance along the Z m axis of the Mechanical Interface 
Coordinate System to the center of gravity of the test load. 
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radial offset. The perpendicular distance from the Z m axis of the 
Mechanical Interface Coordinate System to the center of gravity of the 
test load (see Figure 5.3). 

5.2.4 Test Plane 

The test plane is an unbounded referenced plane within the Manipulator 
working space that is parallel to the (1, 1,-1) plane and passes through the 
working space center point, C w (see Figure 5.4). The test path lies within the test 
plane. 




1 



+X, 




Figure 5.4 - Test Plane Location (ANSL/RIA R1 5.05-1 90) 
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5.2.5 Test Path 



The test path is a sequence of points used to quantify the performance of 
the Manipulator. The test path specified by ANSI allows for the relative 
comparison of accuracy, repeatability, cyclic rate and overshoot between different 
large scale manipulators. All data shall be taken from measurements of motion 
along the test path. The specific measurements that quantify the test paths for 
static and dynamic testing are described in the following sections. 



5.2.5.1 Static (Point to Point! Test Path 

This test path is adopted from ANSI/RIA R1 5.05-1 90. The path is 
located in the test plane and lies along the reference center line E]E 2 (see Figure 
5.2b). Points Ei and E 2 are located at the intersection of the test plane and the 
boundary of the Manipulator working space on a horizontal line that passes 
through the working space center point, C w .. The test path (Figure 5.5) is defined 
as: 



1. A rectangle located in the test plane with Li, Ui, U 4 and L 4 forming the 
comers. 
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2. The test path segment length (Sl) will be 1000 mm, the largest test 
segment recommended by ANSI. The path will contain at least three 
segments. 

3. The rectangular segment side lengths (Dl) will be one-half the path 
segment length (Sl) or 500 mm. 

4. The total length of the test path rectangle is defined as the length of the 
line FiF 2 (3000 mm). The segment end-points are labeled Ui through U4 
along the top (line U1U4) and Li through L 4 along the bottom (line L1L4). 

5. The length of line segments E1F1 and E 2 F 2 will be equal. 

6. The Manipulator shall maintain, where possible, the orientation of the 
Mechanical Interface axis Z m perpendicular to the test plane at all points. 
Where the limiting range of the pivoting motion precludes this (Glass 
1984), the Manipulator shall maintain the Z m axis as near to perpendicular 
to the test plane as possible. 
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5.2.5.2 Dynamic (Path-Related) Test Path 

This test path is adopted from ANSI/RIA R1 5.05-2 92. The path is 
located in the test plane and lies along the reference center line EiE 2 (see Figure 
5.2b). Points Ei and E 2 are located at the intersection of the test plane and the 
boundary of the Manipulator working space on a horizontal line that passes 
through the working space center point, C w .. The test path (Figure 5.5) is defined 
as: 



1 . A rectangle such that the path motions result in the Manipulator 
moving the test load through a large area of the Manipulator’s working 
space. 
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2. The center of the rectangle will be the midpoint, C p , of the center line 
E,E 2 . 

3. The rectangle will be defined (see Figure 5.6) by the four comer points 

(Ri, R 2 , R 3 , Rj). 

4. The segment lengths, S L , shall be 1000mm as recommended by ANSI. 

5. The rectangle will have a height of Sl (1000mm) and a length of 2 Sl 
( 2000mm). 

6. The direction of travel will be clockwise when viewed from the base of 
the Manipulator. 

7. The starting point will be as shown in Figure 5.6. 

8. ANSI recommends a maximum speed of lOOOmm/sec for an 
Sl= 1000mm. 
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Figure 5.6 - Dynamic Test Path (lab tests) 



5.2.6 Working Space Center Point 

The working space center point, C w , of the Manipulator is located at the 
midpoint of the line parallel to the Xi axis whose Z\ axis position will be the 
midpoint of travel in the Zi axis direction (see Figure 5.2). 



5.2.7 Path-Related Evaluation Planes 

ANSI (ANSI/RIA R1 5.05-2 92) introduces the concept of evaluation 
planes to simplify the calculation process by transforming complex three- 
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dimensional path calculations into the intersections between the attained path and 
the two-dimensional evaluation planes (see Figure 5.7). Evaluation planes are 
used to establish discrete locations for the evaluation of path accuracy and 
repeatability. These planes are aligned normal to the test plane and are placed at 
equal linear distances of 1/4 S L apart (250 mm). There are a total of 20 evaluation 
points. Linear interpolation shall be used when an attained point does not lie 
exactly on the evaluation plane (see Figure 4.2). 




Figure 5.7 - Path Related Evaluation Planes 
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5.3 Performance Classes 



ANSI provides different performance classes for optimizing specific 
performance characteristics (for example, repeatability or cycle time). Hardware 
and software adjustments can be made prior to testing for each performance class. 
However, all adjustments must remain constant for every test within the 
performance class. The four classes are: 

• Class I- Standard : To evaluate overall performance without optimizing 
specific parameters. The standard test conditions were outlined in the previous 
section. Per ANSI guidelines, the Manipulator, a hydraulic system, may be 
operated for 1 5 minutes prior to data acquisition. The Manipulator shall not 
experience system overloads or overheating during testing. 

• Class II - Cycle Time (Speed) : To evaluate the Manipulator under optimized 
cycle time conditions. Performance parameters may be varied to enhance cycle 
time performance consistent with intended use but still remain indicative of actual 
cycle time performance. 

• Class III - Repeatability: To evaluate the Manipulator under optimized 
repeatability conditions. Performance parameters may be varied to enhance 
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repeatability performance consistent with intended use but still remain indicative of 
actual repeatability performance. 

• Class IV - Special: To evaluate other specific Manipulator performance 
characteristics. This class is provided to allow testing of characteristics not 
covered in classes I through III. 



5.4 Performance Criteria for Point to Point and Static Testing 

This section outlines the static and point to point performance criteria of 
the Manipulator. The following subsections are derived from ANSI/RIA R1 5.05-1 
90. 



5.4.1 Positional Accuracy (PA) 

ANSI defines static positional accuracy as the statistical measure of the 
spatial deviation between commanded and achieved Manipulator positions. The 
static accuracy will be computed from the data collected during operation of the 
Manipulator under the standard test conditions outlined in section 5.2. The 
recommended procedure is: 
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1 . Calibrate the Manipulator control system. Enter the coordinates of the test 
point into the controller. 

2. Match the Test Equipment Coordinate System with the Base Coordinate 
System at three or more points along the test path. 

3. Enter the commanded poses into the Manipulator controller without physically 
moving the Manipulator. The list of poses will consist of unique commanded 
poses (for example Xi, Yi, Zi, Ai, Bi, Ci). The (Xi, Yi, Zi) coordinates will be 
identical to the test path vertices described in section 5.2.5. 1. The orientation 
coordinates (Ai, Bi, Ci) shall be selected at random from the set of all achievable 
positions. Fifty poses will be input to the controller using multiple visits to each 
vertex. 

4. Command the Manipulator to stop at the selected commanded poses. The 
order of visiting each vertex shall be completely random to provide unique 
approach paths. After the Manipulator has reached stabilization, measure the 
achieved pose in the test equipment coordinates (X,, Y t , Z t ). 

5. Compute the mean (d pa ) and standard deviation (.Spa) accuracy as described in 
Section 4.2.1 and record results. 
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5.4.2 Positional Repeatability 



ANSI defines positional repeatability as the measure of deviation between 
achieved Manipulator positions and the mean of those positions after ordering the 
Manipulator to the same pose N times. The repeatability will be computed from 
the data collected during operation of the Manipulator under the standard test 
conditions outlined in section 5.2. The recommended procedure is: 

1 . Three measurement positions are required and will consist of the Lj, L 2 , and L 4 
positions shown in Figure 5.5. The motion between these measurement positions 
will be along the test path while maintaining the orientation of the mechanical 
interface. 

2. Warm-up drift and the warm-up period are determined from the number of 
readings that elapse, after a cold start, until the system reaches system 
stabilization. Warm-up drift is the positional difference between the first position 
after start-up and the first position after reaching system stabilization. The warm- 
up period is the time this takes measured in minutes. 

3. Calculate and record the Manipulator’s mean positional repeatability (^p) 
and standard deviation (Srep) as described in Section 4.2.3 for a recommended 
sample size (N) of 500 continuous cycles. 
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5.4.3 Cycle Time 



Segment time and traverse cycle time are the two figures recorded for 
cycle time. 

• Segment cycle time is the average time required by the Manipulator to travel 
through one segment of the test path and is recorded in seconds. 

• Traverse cycle time is the average speed attained while completing relatively 
large movements. 

The recommended procedure is: 

1. The test load and test path previously described in section 5.2 shall be used. 

2. During the segment cycle portion of the test, the Manipulator shall be 
programmed to following test path: 

L] — ^ U ] — > U2 — ► L2 — ^ U2 — ^ U3 — ^ L3 — > U3 — > U4 — ^ L4 
The start and end points of each segment are the points labeled L. The 
Manipulator shall reach system stabilization at each point before continuing. 

3. Upon reaching point L 4 , the Manipulator shall return to Li using the following 
return path: L 4 -» U 4 -» Ui -» Li. This is the path for measuring the traverse 
speed. 
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4. The Manipulator shall maintain the orientation of the mechanical interface axis, 
Z m , throughout the test. The maximum deviation for the programmed upper point 
(Ui, U 2 , U 3 , U 4 ) shall be recorded. 



5. Compute segment cycle time as follows: 



segment cycle time = 



Time (L, to L 4 ) 
3 



seconds/segment 



where Time(Lj to L 4 ) is the time required to travel from Li to L 4 in step 2. 
Segment time is the average number of seconds required to move through one 
segment. The total number of segments in this test path is three. 

6. Compute average traverse speed as follows: 



Average traverse speed is the average speed achieved during step 3. 

5.4.4 Overshoot and Settling Time 

Overshoot is measured to quantify the Manipulator capability to make 
smooth and accurate stops. Overshoot is an important performance parameter for 
operations involving large inertia’s, high speed or frequent stops. To measure 



average traverse speed = 



3(S l ) + 2(D l ) 
Time(L 4 toL,) 



meters/second. 
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overshoot the Manipulator is run continuously from Ui — > Li — > Ui — > U 2 — » L 2 
-» U2 — > U3 — ► L3 U3 U 4 — » L 4 -> U4 and then straight to Ui (see Figure 
5.5). Overshoot is equal to the overtravel distance at point Li and is an absolute 
value along the direction of points Ui to Li when the Manipulator approaches 
from point Ui. Overshoot shall be measured for several cycles and the average 
recorded. 

Settling time is measured to quantify the Manipulator’s capability to stop 
quickly at a target point. To measure settling time, the Manipulator is run through 
the same cycle as overshoot. When the Manipulator approaches point Li from Ui 
the position of the test point is continuously measured until system stabilization 
reached. Settling time is measured as the elapsed time from the instance of initial 
crossing into the limit band until the instance when the Manipulator remains within 
the limit band (see Figure 2.4). Repeat the same procedure several times and 
record the average value. 
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5.5 Performance Criteria for Path-Related and Dynamic Testing 



This section outlines the path-related and dynamic performance criteria of 
the Manipulator. The following subsections are derived from ANSI/RIA R1 5.05-2 
92. 



5.5.1 Relative Path Accuracy 

ANSI defines relative path accuracy as the measurement of the distance 
between a reference path and any given attained path. The relative path accuracy 
test utilizes a previously measured path as reference. The reference path is 
acquired by commanding the Manipulator to follow the standard test path at the 
standard test conditions described in Section 5.2. The recommended procedure is: 

1 . Calibrate the Manipulator control system. Enter the coordinates of the test 
point into the controller. 

2. Move the Manipulator such that the test point coincides with the midpoint, C p , 
of the test plane. The C p is used as the origin of the coordinate system defining the 
reference path. The Test Equipment Coordinate System is aligned with the test 
plane per Section 10.2.3.3 of ANSI/RIA R1 5.05-2 92. 
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3. Program the Manipulator to follow the test path described in Section 5.2 at a 
speed of lOOmm/sec. Measure the (Xr J( Y^. Zr,) points (see Figure 4.1) for each of 
the m evaluation points shown in Figure 5.6. Use linear interpolation to calculate 
the intersection of the relative reference path with the evaluation plane (see Figure 
4.2). These values will be used as the reference path to measure relative path 
accuracy. 

4. Program the Manipulator to follow the standard test path at 50% and 100% of 
maximum speed. Measure the attained (Xrj, Yr,, Zx) coordinates for each 
evaluation point. 

5. Calculate the maximum path accuracy (ACrel) and the average path accuracy 
(ACrel) as described in Section 4.2.2 and record the results. 



5.5.2 Path Repeatability 

ANSI defines path repeatability as the measure of the closeness between 
multiple attained paths. Path repeatability is measured using the same test 
procedure described in Section 5.5.1 above. The difference between the measured 
path accuracy and path repeatability is that path repeatability uses the average of a 
path traversed n times (see Section 4.2.4). Calculate the maximum path 
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repeatability (PR) and average path repeatability (PR) as described in Section 
4.2.4. 



5.5.3 Path Speed Characteristics 

ANSI defines four terms to quantify path speed characteristics: 

1 . Path speed accuracy (AS) : the difference between the programmed speed and 
the mean value of the attained speed during n traversals along the test path AS is 
expressed as a percentage of the programmed speed: 



s - s 1 n 1 m 

AS(%)= xlOO where,! = —Vs,, and s, = — Ts 

n m Ti' 



Here. 

s p is the programmed speed; 

Sjj is the attained speed for the /th cycle and y'th evaluation point; 
m is the number of evaluation points; 
n is the number of cycles; 

s is the mean speed for one traversal of the test path. 
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2. Path speed Repeatability (RS): the closeness of agreement of the speeds 
attained for the same programmed speed. Using the same procedure as path speed 
accuracy: 



RS(%) = 




x 100. 



3. Path speed fluctuations (FS): the maximum deviation in speed for a single 
traversal of an attained path at constant speed conditions. Using the same 
procedure as path speed accuracy: 



n m m 




4. Path acceleration time (TS): the time to move from zero speed to programmed 
speed. The time interval is measured from the first detection of motion to the 
point where the Manipulator attains the average speed within the tolerance defined 
by the path speed fluctuation (FS). 
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5.5.4 Cornering Overshoot 



ANSI defines comer overshoot (CO) as the maximum deviation past the 
target. It is measured as the largest deviation outside the reference path after the 
Manipulator has “passed” a corner. The value of CO can be calculated for each of 
the three corners traversed in the standard test path. The equation for CO is: 

CO = max V(X. l -X, l ) ! +(Y, ( .Y, k ) J +(Z, l .Z r> ) : . 



Where: 

• X a u, Y ak , Z ak are the position coordinates on the attained path, 

• X r k, Y r k, Z r k are the coordinates along the reference path; 

• A: is the subscript number for each of the discrete data points along the path and 
is dependent upon the test equipment sampling rate (see ANSI/RIA R1 5.05-2 
92 for further explanation). 
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CHAPTER SIX 



CONCLUSIONS 



This report presents various methods of performance qualification for large 
scale construction manipulators for the purpose of ascertaining physical 
specifications. The report defines the most important static and dynamic 
performance criteria and presents a method for evaluating them. These criteria are 
accuracy, repeatability, overshoot, settling time and cycle time. 

A set of meaningful benchmark tests are presented to gauge the overall 
static and dynamic performance of different large scale manipulators in order to 
provide a means of relative comparison. These performance tests are based on the 
American National Standard ANSI/RIA 15.01-1 90 for Point-to-Point and Static 
Performance and ANSI/RIA 15.01-2 92 for Path-Related and Dynamic 
Performance Evaluations of industrial robots and robot systems. 

Statistical methods are included for calculating the probability that the 
Manipulator will achieve a desired target within a calculated standard deviation. 
The formulas for computing position accuracy, repeatability and their respective 
standard deviations are reviewed for both static and path-related output. 
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Finally, the report reviewed the application of human factors engineering 
and presented a method to quantify potential improvements to the human-machine 
interface. 

The large scale manipulator owned by the University of Texas is a very 
adaptable machine. Its eight degrees-of-ffeedom, sixty-five foot working radius 
and multi-functional potential make it a noteworthy test bed for developing 
enhanced control and performance testing strategies. With the application of these 
benchmark tests, a means of comparing the relative strengths and weaknesses of 
different controllers and manipulators can begin. 
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